Telephone based tele-health apparatus

ABSTRACT

A tele-health apparatus includes a telephone having a microphone, an auscultation piece to acquire sounds, and a solid medium acoustically coupling the auscultation piece to the microphone. The auscultation piece is part of a stethoscope, and the solid medium is a windpipe of the stethoscope. The tele-health apparatus also includes an otoscope operable to be disposed in front of a camera of the telephone. A clip holds the stethoscope and the otoscope, and is fixed to the phone. Software modules installed in the telephone enable the tele-health apparatus to engage a user in a two-way audio and/or video consultation with a physician at a remote device in real-time.

PRIORITY CLAIM

The instant patent application is related to and claims priority fromthe following two provisional US patent applications:

-   -   A) Entitled, “Synchronous Tele-Health Diagnostic System        Integrated With Smart Devices”, Ser. No. 63/072,961, Filed: 1        Sep. 2020, and    -   B) Entitled, “Synchronous Tele-Health Diagnostic System        Integrated With Smart Devices”, Ser. No. 63/118,751, Filed: 27        Nov. 2020,        both of which are incorporated in their entirety herewith to the        extent not inconsistent with the description herein.

BACKGROUND Technical Field

Embodiments of the present disclosure relate generally to healthcareapparatuses, and more specifically to a telephone-based tele-healthapparatus.

Related Art

Tele-health apparatuses enable doctors and other medical personnel toremotely serve persons requiring medical assistance. Telephones refer todevices which enable two or more persons to conduct voice and/or videocalls. While telephones were wire-based in early evolution of thetechnology (referred to as Plain Old Telephone Systems, POTS),telephones are now available in the form of mobile/smart phones whichcommunicate wirelessly on 5G/4G/3G, WiFi etc., communication standards.

Because of the ubiquity of telephones, there is a constant demand toprovide tele-health apparatuses which leverage the capabilities oftelephones.

BRIEF DESCRIPTION OF THE VIEWS OF DRAWINGS

Example embodiments of the present disclosure will be described withreference to the accompanying drawings briefly described below.

FIGS. 1 through 4, 5A and 5B are diagrams of different views of atelephone-based tele-health apparatus in an embodiment of the presentdisclosure.

FIG. 6A is an exploded diagram illustrating the parts of an otoscope ina telephone-based tele-health apparatus in an alternative embodiment ofthe present disclosure.

FIGS. 6B and 6C respectively show two different views of an otoscope inassembled form, in the alternative embodiment.

FIG. 7 is a diagram illustrating a fiber-holder ring used in an otoscopein a telephone-based tele-health apparatus in an alternative embodimentof the present disclosure.

FIG. 8 is a block diagram of several application modules executed in atelephone-based tele-health apparatus, in an embodiment of the presentdisclosure.

FIGS. 9A through 9L are illustrations of example screens displayed on atelephone-based tele-health apparatus enabling interaction of a userwith features provided by the apparatus.

FIG. 10 is a block diagram of a telephone implemented according toseveral aspects of the present disclosure.

In the drawings, like reference numbers generally indicate identical,functionally similar, and/or structurally similar elements. The drawingin which an element first appears is indicated by the leftmost digit(s)in the corresponding reference number.

DETAILED DESCRIPTION 1. Overview

According to an aspect of the present disclosure, a stethoscope and anotoscope are fitted to a telephone. The telephone may, for example, be areadily available, off-the-shelf device such as a mobile phone. Thestethoscope is connected to the microphone port of the telephone, andthe otoscope is fitted to the telephone such that it couples light to acamera in the telephone. The stethoscope may be a passive mechanical andacoustic attachment (to the telephone), and the otoscope may be apassive mechanical and optical attachment (to the telephone). The term‘passive’ with respect to a part implies that there are no components inthat part that require electric power for its operation in accordancewith features described herein.

In conjunction with corresponding application modules executable in thetelephone, the integrated tele-health apparatus (telephone plusstethoscope plus otoscope plus application modules) enables a two-wayaudio and/or video consultation between a user and a remote medicalprofessional or physician. Synchronous, real-time diagnostic dataobtained by the apparatus via the stethoscope and otoscope (operated bythe user as directed by the remote physician) are made available to thephysician, who can then prescribe the required treatment and medicines.Thus, the tele-health apparatus can provide a tele-health solution thatmay be equivalent to primary care physician in-office examination.

Several aspects of the present disclosure are described below withreference to examples for illustration. However, one skilled in therelevant art will recognize that the disclosure can be practiced withoutone or more of the specific details or with other methods, components,materials and so forth. In other instances, well-known structures,materials, or operations are not shown in detail to avoid obscuring thefeatures of the disclosure. Furthermore, the features/aspects describedcan be practiced in various combinations, though only some of thecombinations are described herein for conciseness.

2. Telephone-Based Tele-Health Apparatus

FIGS. 1 through 4, 5A and 5B are diagrams of different views of atelephone-based tele-health apparatus (Apparatus 100) in an embodimentof the present disclosure. In the embodiment, a smart phone (110) isused as the telephone. As is well-known in the relevant arts, a smartphone is a mobile device that combines cellular and computingcapabilities into one unit. A smart phone typically contains a mobileoperating system, which enables applications to be loaded on the smartphone, thereby enabling features such as for example, web browsing,audio and video players, cameras, etc., in addition to traditional phonefunctions such as voice calls. However, in other embodiments, otherdevices (which permit voice calls) such as tablet PC (personalcomputer), iPads, internet-connected chrome books, computers, IoT(Internet of Things) enabled devices as well as wired telephones, IP(internet protocol) phones can be used instead. FIG. 1 is showncontaining smart phone 110, clip 120, stethoscope 130, otoscope X-Yadjuster 140 and otoscope 150.

Smart phone 110 represents an off-the shelf mobile telephone with anoperating system, and supports addition (installation) of applicationmodules, and supports circuit-switched and/or packet-switched voiceand/or video calling. Smart phone 110 is equipped with one or moremicrophones, as well as one or more cameras. A microphone port 180 of amicrophone is shown in FIG. 5A. Thus, a voice signal received atmicrophone port 180 is converted into an electronic signal suitable fortransmission to the other end, where another phone or a device such as acomputer, tablet or otherwise reproduces the voice signal from theelectronic signal. A camera port 160 of a camera on the back-side ofsmart phone 110 is shown in FIG. 1 . Camera port 160 is used inconjunction with the otoscope, as described below. Phone 110 also has afront-side camera, which is used during a video call, thus enabling theremote physician to “see” the patient/user.

Clip 120 is a mechanical assembly or harness and is designed to grip theouter edges and back of smart phone 110 using, for example, springaction or a tightening screw. Clip 120 is more clearly shown in FIGS. 1and 3 . Stethoscope 130 and otoscope 150 are attached (as noted below)to clip 120, and thereby are secured to smart phone 110. Clip 120 issized such that it does not interfere with device controls on smartphone 110. Clip 120 may also be designed so as to meet the device (smartphone 110) manufacturer's attachment criteria, if any.

Stethoscope 130 is made up of auscultation piece (or chest piece) 130A,wind pipe 130B and coupling piece 130C. Auscultation piece 130A is inturn made of a flexible diaphragm, an acoustic chamber and a ring or nutto elastically fasten the flexible/elastic diaphragm to the acousticchamber. FIG. 4 shows the acoustic chamber 130A-1 more clearly. FIG. 5shows flexible diaphragm 130A-2 and ring or nut 130A-3 more clearly.Acoustic chamber 130A-1 may be made, for example, of metal or rigid hardnon-porous plastic. Auscultation piece 130 is used for acquiring sounds,such as for example, heart and lung sounds, or in general, sounds insidea patient's body. Auscultation piece 130 may need to be placed close tothe source of the sound.

Wind pipe 130B carries acoustic waves from the auscultation piece 130Ato the microphone port/opening of smart phone 100, and may be a hollowrubber tube, for example (solid medium, in general). One end of the windpipe 130B is connected to the auscultation piece 130A, while the otherend is connected to coupling piece 130C. Coupling piece 130C couplesauscultation audio waves to smart phone 110's microphone outlet/portwhile securing the wind pipe through a compression fit. For cushioningand optimal air coupling into the microphone hole/holes one or moresuction cups or soft rubber couplers is/are used. This coupling allowsfor user voice to be captured without significant attenuation and theuser can continue regular audio (via the microphone) and/or video/visualcommunication without interruption or significant degradation.

Auscultation piece 130 is secured to clip 120 (when not in use) bydouble-sided tape or glue or a hook. Alternatively, auscultation piece130 can be secured to clip 120 using a screw, hook, temporary stowingmechanism or be permanently molded as part of clip 120, in which casethe chest piece can't be removed but provides an advantage ofcompactness. Sounds received via stethoscope 130 and the microphone portof smart phone 110 are processed and/or transmitted (in real-time, forexample) by application modules in smart phone 110 to a remote device(such as, for example, another phone) for analysis by a doctor(physician). The application modules can also simultaneously play thesounds on a local speaker/earphones of smart phone 110, or record themlocally in smart phone 110. It is noted here that, since microphone ofphone 110 is employed for receiving heart auscultations, an ear phonewithout a microphone (or microphone disabled) needs to be used by theuser. Alternatively, the tele-health application can be designed toforce the software that controls the (earphone+external microphone)headset to use the phone's microphone port 180 and not the externalmicrophone provided with the headset.

Although windpipe 130B in the Figures is shown to be rather short, thelength of windpipe 130B can be much longer in practice. In anembodiment, the length of windpipe 130B is at the maximum 300centimetres with no minimum length limit. Due to such length, the usercan simultaneously view the physician on display of phone 110, listenand converse with the physician, allow the physician to see the user viathe front-side camera of phone 110, as well as simultaneously operatestethoscope 130 to capture heart and lung auscultation audio. Such afacility enables easy use by the user especially if the user is elderly,differently-abled person, or a sick patient, since the user now does notneed to juggle with two or more devices simultaneously causingasynchronous and confusing data streams for the physician as well as thepatient, and thereby deteriorating the diagnostic quality. Also, useranxiety associated with losing contact with the remote physician isabsent in this embodiment. However, in an alternative embodiment, thewindpipe 130B is eliminated and chest piece 130A is directly coupled tocoupling piece 130C.

Otoscope 150 is made up of speculum 150A and lens holder 150B. Speculum150A and lens holder 150 are attached to otoscope X-Y adjuster 140,which in turn is pivoted via pivot mechanism 121 on clip 120. Lensholder 150B houses a magnifying lens, and the focal length of the lensis adjustable by a screw to enable light from the viewed source to befocused onto camera port 160. Otoscope 150 can be swiveled via the pivotmechanism 121, using radial co-ordinates to be positioned in front ofcamera port 160. An alternative embodiment of the pivot mechanism 121employs a Cartesian X-Y translation frame. FIG. 3 shows otoscope 150positioned in front of the camera port. Otoscope 150 enables auser/patient of smart phone 110 to obtain videos or still pictures of,for example, ear, nose and throat, etc., of a user/patient, bypositioning speculum 150A in the part to be viewed. Videos and stillpictures (i.e., images) obtained using otoscope 150 and camera port 160are processed and/or transmitted (in real-time, for example) byapplication modules in smart phone 110 to a remote device (such as, forexample, another phone, computer or a smart communication device) foranalysis by a physician. The application modules can also simultaneouslyrender the videos and/or still pictures on a display screen of smartphone 110, or record them locally in smart phone 110.

In an alternative embodiment of the present disclosure, the otoscopefunctionality is augmented to also use the flash LED (light emittingdiode) of smart phone 110 in conjunction with a light guide toilluminate the object desired to be viewed and analyzed. FIG. 6A is adiagram showing an exploded view of the otoscope in the alternativeembodiment. When in use, otoscope 600 shown there is positioned suchthat light collected by the otoscope impinges on a camera port of smartphone 110, such as camera port 160 (FIG. 1 ). Similar to otoscope 130,otoscope 600 may also be fitted to swivel on a clip such as clip 120 asillustrated in FIGS. 1-4, 5A and 5B. Alternatively, otoscope 600 may beattached to smart phone 110 using double-sided sticky tape (removable)or elastic band mechanisms.

Referring to FIG. 6A, otoscope 600 shown there is made of parts speculum610, optic fiber holder ring 620, lens 630, tunnel 640 and cylinder 660.Speculum 610 functions similar to speculum 150A of FIG. 1 to collectlight from a viewed object (ear, nose, etc.). In addition, speculum 160has fiber guide paths on it through which optic fibers can channel lightfrom a light source such as the flash LED of smart phone 110 onto theobject to be viewed. Example guide paths 615, which would be in thespeculum, are shown in FIG. 6A.

Fiber holder ring 620 contains holes through which individual opticfibers are passed and thus secured in place when the components of FIG.6A are assembled. The holes are aligned with the guide paths in speculum610. The number of fibers, fiber diameter and therefore the holes inring 620 may be determined based on the size of the flash LED. Fiberholder ring 620 is shown separately and in greater detail in FIG. 7 , inwhich only one hole 711 is marked. The entire fiber bundle made ofindividual fibers (such as 710) are collected (bundled together, asindicated by marker 720) and optically coupled to the flash LED throughcompression, by which the geometric plane containing the bundled fibers'faces (720 of FIG. 7 , for example) coincides with the geometric planethat contains the LED flash outer surface without any air gap, and thecenter of the fiber bundle approximately coincides with the center ofthe flash to allow for optimum coupling of the light emanating from theflash to enter the bundled fibers.

Cylinder 660 attaches to ring 620 at one end, and the other end isattached to an X-Y adjuster (not shown) such as adjuster 140 of FIG. 1 .Alternatively, the X-Y adjuster is not used, and cylinder 660 isdirectly attached to camera port using suitable mechanisms (such as forexample double-sided sticky tape), such that the camera port is rightbelow the bottom part (690) of cylinder 660. Lens 630 is used formagnification, and is fitted to be within cylinder 660 and concentric toit.

Tunnel 640 receives the fiber bundle from ring 620, and routes thebundle to the flash LED of the camera (indicated in FIG. 7 by marker720. Tunnel 640 may be designed using flexible optical fiber/fibers orlight pipes. The flash lens (not shown) of the camera on smart phone 110would be at the bottom of tunnel 640. Slot 650 is an adjustmentmechanism to adjust the center of the light-carrying fiber bundle tocoincide with that of the flash, while achieving one-to-one lightcoupling between the flash and fibers. This mechanism allows foradjusting for flash location differences in different smartphones. Theentire otoscope assembly can be pivoted around the pivot point (similarto pivot point 121 of FIG. 1 ) or be mounted on a Cartesian X-Y stage toallow the user to approximately align the back-side camera center withlens tower center (i.e., cylinder 660). Tab 640A and 640B (FIG. 6C) canbe pulled in and out of the assembly so as to allow for flash-fibercenter alignment. The light carrying fiber bundle (e.g., 720 shown inFIG. 7 ) is tied and routed through a hole inside tab 640B to allow foralignment of the plane of the fiber bundle with that of the flash lens.

FIGS. 6B and 6C depict two different views of otoscope 600 in assembledform.

It is noted here that the otoscope (150 or 600) contains opticalattachment filters (attached to the otoscope/optical attachment) toconvert IR to visible wavelength. Thus, in FIG. 1 , the filter would becontained in the lens holder 150B along with the lens. Similarly, inFIG. 6A, the filter would be contained in cylinder 660 proximal to lens630. The reason for the addition of the filters is that a smart phonecamera such as that of phone 110 may not detect IR (infra-red light),and IR from the human body has rich information. The optic filtersconvert IR into useful visible light spectrum. Ambient visible light isblocked using black colored soft rubber blockers coupled to bottomsurface 690, so as to prevent ambient light from getting into theotoscope, as well as to avoid making scratches on the smartphone cameralens, body and flash.

It may be appreciated that apparatus 100 being a single apparatus(single piece), may lend to easy use by a patient. It is noted here thatmost of the existing devices or systems available for point-of-carediagnostics appear to require additional hardware to be implementedwithin the device, and suffer from redundant communicationinfrastructure and/or asynchronous streams of the two devices makingdiagnosis confusing and inaccurate. Alternatively, the prior systemsappear to require a dedicated, redundant communication infrastructure inorder to function and provide point-of-care diagnostics. Suchrequirements represent usability problems where the caregiver or apatient needs at least two different devices in order to get thepoint-of-care diagnostics information relayed to physician or a healthcare provider, albeit asynchronously. Using two devices is problematicin the case where patient is either non-cooperative or constrained andcauses diagnostic inaccuracies. For example, if a parent or a caregiverfor a senior, differently abled person, or a sick patient makes atele-health call to a physician, from a usability perspective, they haveto juggle with two devices simultaneously causing asynchronous andconfusing data streams for the physician as well as the patient, thereby deteriorating the diagnostic quality. In cases where there is nopoint of care diagnostic equipment available, the physician has to do adiagnosis and prescribe medication based solely on video stream and orpatient interview. These disadvantages make current devices or systemsvery ineffective to provide quality health care to the patient.

Apparatus 100 on the other hand enables a user to view as well as speakwith the physician at the remote device even while handling stethoscope130. Also, when using the otoscope, phone 110 maintains audio as well asvideo contact (video contact via front-side camera) with the physicianintact by automatically starting the speaker of the phone when the useris using the otoscope (for example for examination of the ear), asopposed to having the audio channeled through the headphones which theuser may find difficult to use simultaneously with the otoscope.

Smart phone 110 contains (or can be loaded with) application modulesthat integrate and synchronously (in real-time) transmit diagnosticinformation obtained via stethoscope 130 and otoscope 150/600 to aremote device. Smart phone 110 also contains application modules fordetermining pulse rate, temperature, blood pressure and pulse oxygencontent of the user from images/videos obtained from a finger-press oncamera port 160. In combination with corresponding software on a remotedevice/terminal, apparatus 100 provides a two-way audio and/or videoconsulting capability for remote diagnosis, archival, medicineprescription (or treatment advice) and billing. The operation of somethe application modules in smart phone 110 is described next.

3. Application Modules

FIG. 8 is a block diagram of several application blocks/modules that canexecute in smart phone 110. The specific blocks are shown merely by wayof illustration, and there can be many more application blocks that canexecute in smart phone 110. Further, the operations of some of theblocks can be combined in a single block also. The term “softwaremodule” as used herein can refer to a single block or a combination ofmultiple blocks. Further still, a mobile operating system (OS) may bepresent in phone 110.

It is noted here that one or more of the blocks of FIG. 8 may rely onother application modules (not shown, but contained in the tele-healthapplication—which refers to the combination of all the modules as awhole) for decision making, forwarding data to a next module etc.Accordingly, it is assumed in the following description that thetele-health application contains an application module control block(AMCB) that generates control signals for the blocks of FIG. 8 (whereneeded). One or more of such control signals may be generated based onthe user's interaction or inputs to the tele-health application. As anexample, when a user presses button 941 in the screen shown in FIG. 9H,the tele-health application ‘knows’ that the user is going to obtainheart auscultation audio. Therefore, the AMCB sends a signal tomicrophone interface block 805 to forward the audio data received onpath 801 to heart auscultation audio block 810, and not to lungauscultation audio block 815.

In an embodiment, one or more of the blocks of FIG. 8 may be designedaccording to WebRTC (Web Real-Time Communication) technology to enablereal-time or synchronous exchange of various diagnostic and otherinformation between phone 110 and a remote device. As is well known inthe relevant arts, WebRTC refers to a free, open-source project thatprovides web browsers and mobile applications with real-timecommunication (RTC) via simple application programming interfaces(APIs). Thus, for example, microphone interface block 805 and camerainterface block 830 may be implemented using getUserMedia API (providedby WebRTC), and transmit block 890 may be implemented usingRTCPeerConnection API (also provided by WebRTC).

Referring now to FIG. 8 , microphone interface block 805, heartauscultation audio block 810, lung auscultation audio block 815 andaudio play block 820 operate on digitized audio data. Microphoneinterface block 805 receives digitized audio data from microphone portof phone 110, and via audio hardware (which includes Analog-to-DigitalConverter (ADC)) of phone 110 when stethoscope 130 is used to obtainaudio information. Microphone interface block 805 may be implementedusing application programmers interface (API) function call provided bythe operating system in phone 110. Microphone interface block 805forwards the received audio data to the corresponding one of blocks 810and 820, depending on whether the audio data represents heartauscultation audio or lung auscultation audio. Microphone interfaceblock may make such determination based on inputs from the applicationmodule control block (AMCB) as noted above. Referring to FIG. 1 , heartor lung auscultation audio is obtained by the user placing auscultationpiece 130A to either the chest or lung area of the user. The user may beguided via phone 110 by a physician at the remote end to placeauscultation piece 130A appropriately.

Heart auscultation audio block 810 applies digital filtering to removenoise from the received heart audio data to obtain filtered heart audiodata. Based on inputs from the application module control block noteabove, heart auscultation audio block 810 may forward the filtered datato one or more of audio play block 820, storage block 880 and transmitblock 890. The inputs from the application module control block may bebased on whether real-time local play, remote play, local storage,remote or cloud storage, archival and retroactive play are required tobe performed—these could again come from user inputs to the tele-healthapplication. In an embodiment of the present disclosure, heart audioauscultation data are simultaneously (synchronously) played locally inphone 110 as well as in a remote device (after concurrent or delayedtransmission as required by the physician at the remote device),enabling real-time and/or asynchronous diagnosis by the physician at theremote device.

Lung auscultation audio block 815 applies digital filtering to removenoise from the received lung audio data to obtain filtered lung audiodata. Based on user inputs or inputs from the application module controlblock, lung auscultation audio block 815 may forward the filtered datato one or more of audio play block 820, storage block 880 and transmitblock 890. The inputs from the application module control block may bebased on whether real-time local play, remote play and local storage arerequired to be performed. In an embodiment of the present disclosure,lung audio auscultation data are also simultaneously (synchronously)played locally in phone 110 as well as in a remote device (afterconcurrent or delayed transmission as required by the physician at theremote device), enabling real-time and/or asynchronous diagnosis by thephysician at the remote device.

It is noted here that heart sounds have a frequency range ofapproximately 50-60 Hz. The digital filter(s) used within heartauscultation audio block 810 may use arbitrary magnitude, low pass andhigh-boost filtering optimized to capture the frequencies of heartaudio. High-boost filtering refers to emphasizing (boosting) of highfrequency components in the audio without eliminating low frequencycomponents. The digital filter(s) used in lung auscultation audio block815 are designed to optimize and boost frequencies from approximately300 Hz to approximately 550 Hz (respiratory frequencies). The filters ofblocks 810 and 815 can implemented as fixed FIR (Finite ImpulseResponse) and IIR (Infinite Impulse Response), or adaptive filters.

Audio play block 820 forwards digital audio data on path 821 to ahardware audio subsystem (which may include digital-to-analog converter(DAC), power amplifier and speaker) in phone 110. Audio play block 820may be implemented using application programmers interface (API)function call provided by the operating system in phone 110.

Storage block 880 receives data representing audio or video/images froma corresponding block of FIG. 8 , formats the data according tocorresponding storage formats, and forwards, on path 881, the formatteddata to a storage device of phone 110. Storage block 880 may beimplemented using application programmers interface (API) function callprovided by the operating system in phone 110.

Transmit block 890 receives data representing audio or video/images froma corresponding block of FIG. 8 , formats the data according tocorresponding transmission formats (including packetizing), andforwards, on path 891, the formatted data to a transmitter of phone 110.Transmit block 890 may be implemented using application programmerinterface (API) function call provided by the operating system in phone110.

Camera interface block 830, otoscopy processing block 835, BP/heartrate/temperature/oximeter block 840, Optical Character Recognition (OCR)block 845 and display block 860 operate on digitized video or image(still picture) data.

Display block 860 forwards, on path 861, to display hardware in phone110, video/images received from one or more of blocks 845, 835 and 840on respective paths 856, 836 and 846. The display hardware renders thevideo/images on a display device (e.g., screen of phone 110). Displayblock 890 may be implemented using application programmer interface(API) function call provided by the operating system in phone 110.

Camera interface block 830 receives, on path 831, digitized video orimage data from camera port 160 and via video hardware (which includesanother (ADC)) of phone 110. The video/image may be obtained usingotoscope 150, from a finger press of the user's finger on the cameraport, or by focusing the camera of phone 110 on display areas of one ormore external diagnostic devices such as a digital or analogthermometer, digital or analog blood pressure monitor, digital or analogoximeter, etc.

When otoscope 150 is used, speculum 150A of otoscope 150 is placed (fore.g., by the user) at the body part to be viewed (e.g., ear, nose orthroat), and X-Y adjuster 140 is adjusted so that the speculum 150A andlens (in lens holder 150B) are aligned with camera port 160. When afinger press of the user is to be captured, the user adjusts X-Yadjuster 140 to cause speculum 150 and the lens to be moved away fromthe camera port. When display of an external diagnostic device isdesired, the user focuses camera port of camera 110 on the display areaof such external diagnostic device.

Camera interface block 830 may be implemented using API function callprovided by the operating system in phone 110. Camera interface block830 forwards, based on inputs from application module control block(noted above), the received video/image data to the corresponding one ofblocks 835, 840 and 845, depending on whether the video data is capturedusing otoscope 150, finger press of user or an external diagnosticdevice

Otoscopy processing block 835 filters the received video/image dataobtained using otoscope 150, and forwards the filtered data to one ormore of display block 860, storage block 880 and transmit block 890depending on corresponding inputs received from application modulecontrol block (noted above). The inputs from the application modulecontrol block may be based on whether real-time local play, remote playand local storage are required to be performed, which in turn may bebased on user inputs to the tele-health application. In an embodiment ofthe present disclosure, the filtered data from otoscopy processing block835 are simultaneously (synchronously) rendered locally on display block860 of phone 110 as well as in a display of a remote device (afterconcurrent transmission to the remote device), enabling real-timediagnosis by the physician at the remote device.

OCR block 845 receives images (captured by the back-side camera, forexample) of display areas of external diagnostic devices (as notedabove), and operates to extract text, numbers, symbol etc., in theimages. OCR block 845 forwards the extracted information to one or moreof display block 860, storage block 880 and transmit block 890 dependingon corresponding inputs received from application module control block(noted above). For example, the external device could be a digitalthermometer. The user can obtain his body temperature using the digitalthermometer, and OCR block 845 can receive the image of the reading onthe display area of the digital thermometer, and extract the temperaturevalue from it.

BP/heart rate/temperature/oximeter block 840 operates to determine theblood pressure (BP), heart rate, body temperature and blood-oxygensaturation level of the user from video/images of a finger press of theuser on camera port 160.

The signal processing algorithms in block 480 typically select a regionof interest around the approximate center of the field of view (FOV) (inthis case, approximately the center of the finger pressed). The size ofthis FOV region directly affects the computational complexity, and anoptimum for the size exists which provides diagnostic quality results,with diminishing returns observed with increasing FOV size. Block 840extracts the average values for Red/Green/Blue (RGB) channels at a highframe rate, as a function of time. The manner in which each ofparameters BP, heart rate, body temperature and blood-oxygen saturationlevel is determined is now described.

As noted above, block 840 operates on a sequence of RGB images generatedby a finger press of the user on camera port 160. Block 840 creates afixed volume or region of interest (ROI) within each image of thesequence of images. Each image of the sequence is a function of lightintensity I, as well as the contents of the ROI on a macroscopic level.Each of the images contains red, green and blue (RGB) values of thesensor (e.g., CMOS) outputs of the camera, and block 840 extracts theaverage values for Red/Green/Blue (RGB) channels at high frame rate, asa function of time.

The dermal (skin) component of the obtained sequence of images (or morespecifically the RGB values of the images) as a function of time is aconstant for any given scenario, and the light level variations in thesequence of images is due to the effect of blood flowing in and out ofthe ROI as the heart goes through the systolic and diastolic phases.Therefore, the light level variations in the sequence of images are alsoa function of (i.e., correlated with) the user's blood pressure. Thus,the ROI is representative of the heart function and correlates with theblood pressure as a function of time.

The light level values at the peak of a systole are also a function ofoxygen content in the blood during the peaks of the systoles. Block 840corrects for the oxygen content variation by first determining the peaksof the pulses (systolic) corresponding to heart rate based on redintensity values in each image. A systolic peak would correspond tomaximum red intensity among all images other than those representingother systolic peaks. Then, block 840 estimates the oxygen contentcomponent by subtracting from images representing the systolic peaks,the average value of red intensities in the images corresponding tosystolic peaks and extracting oxygen content from flat field-correctedintensities. As is well known in the relevant arts, flat-fieldcorrection is a technique used to improve quality in digital imaging bycancelling the effects of image artifacts caused by variations in thepixel-to-pixel sensitivity of the detector (camera) and by distortionsin the optical path. The blood oxygen content component thus obtained(or alternatively obtained by averaging estimates of blood oxygencontents over several iterations) indicates the blood-oxygen saturationlevel.

The heart rate is determined by averaging the period of detected peaks(noted above), and by obtaining an inverse of the period. The bodytemperature is correlated with heart rate elevation and is anage-dependent factor. Block 840 employs well-knownage-heart-rate-temperature relationship to obtain the patient'stemperature.

Continuing the description with respect to BP determination, oncecorrected for blood oxygen content, the only remaining dependentvariable is the BP. Block 840 processes multiple successive frames(images) (as an example, the camera may provide 30 frames/second orhigher frame rates) to obtain blood volume (correlated with red valuesin the images) around (i.e., red intensities in image immediately before(with respect to the sequence of images) and immediately after imagerepresenting a systolic peak, and red intensities in image immediatelybefore and immediately after image representing a diastolic peak) todetermine the rate of volumetric change of blood as a function of time,i.e., dV/dt corresponding to systolic peaks and diastolic peaks.

A region of interest (ROI) is drawn on the approximate center of thefinger's image (i.e., in each of the sequence of images). This ROI orthe Field of View (FOV) is an arbitrarily small area on thetwo-dimensional image. The blood flow rate (dV/dt) in and out of thisFOV as volumetric change (dV) as function of time (dt) is determined byplotting the corrected (as noted above) red intensities as a function oftime by analyzing successive frames thus acquired with its correspondingtime stamp (dt).

The rates of volumetric change dV/dt corresponding to systolic peak anddiastolic peak are then converted to respective values of dP/dt, i.e.,rate of change of pressure with time, through a linear transformationassuming first-degree approximation for a very small part of the pressedfinger, further miniaturized by arbitrarily choosing a smaller subset ofthe image called field-of-view as described earlier. The lineartransformation constant K is a function of sensor (camera sensor)response characteristics and can vary from one device to another. Thisvalue, i.e., constant K, is determined using a calibration step withknown BP.

The BP determined as noted above may contain error components due to thelight intensity (lumens) used for obtaining the sequence of images ofthe finger press and camera resolution variations (across differentcameras). Such error components are corrected using commonly knownflat-field correction method, in which the sensor (camera elements)response is measured under uniform illumination to establish a baseline.This baseline accounts for any system induced variations, notcontributing to the signal of interest and is subtracted from themeasured signal as a flat-field correction.

The parameters and signals, such as heart auscultation and lungauscultation audio, images acquired by the otoscope, BP, temperature,blood-oxygen content, etc., as noted in detail above may be termed asdiagnostic data.

Thus, various parameters and diagnostic data related to the user'shealth are obtained and transmitted in real-time to a physician at aremote terminal. The description is continued with a brief illustrationof an example user interface provided by application modules (other thanthose of FIG. 8 ) on smart phone 110 to enable a user/patient tointeract with a physician at the remote terminal, obtain diagnosticmeasurements, etc.

4. User Interface

Once a tele consultation is setup, the patient and physician connectwith each other by opening an application (tele-health application notedabove) on their respective smart phones. The patient upon physician'sverbal or non-verbal (textual) or sign (video) instructions will clickon menu items within the applications (examples noted below) to acquireand synchronously relay the diagnostics information from thepoint-of-care to a remote physician. Different diagnostic informationwill be acquired either serially or in parallel based upon the medicalneeds as seen fit by the health care provider or physician. Upon thecompletion of diagnostics acquisition, the prescription will be sent tothe patient on phone 110, or the pharmacy of their choosing. At the endof the consultation billing related exchanges will happen between thepatient and healthcare provider.

FIGS. 9A-9L are example screens provided by the tele-health applicationthat a user interacts with. In each of the screens illustrated, the‘cancel’ button at the top-left of the screen cancels the currentoperation(s), and the user is presented with the next appropriatescreen.

FIG. 9A is a screen that is presented on the display of phone 110 to auser upon launching the tele-health application. The user can enter hisemail ID in box 901 and password in box 902, and then press button 903to sign-in or register (if not yet registered). Upon signing-in, theuser is presented with the screen shown in FIG. 9B.

In FIG. 9B, pressing button 907 allows the user to fix an appointmentwith a physician, and upon pressing button 907, the user is presentedwith screen of FIG. 9E. Referring to FIG. 9E, the user can select one ofthe physicians indicated by reference numerals 920, 921 and 922.Assuming the user selects Lisa Su (920), the user is then presented withthe screen of FIG. 9F, in which the user can enter the date (925) andtime (926) for the appointment. Pressing the set button 927 generatesthe screen of FIG. 9G, which displays the physician's name (931), date(932) and time (933) of the appointment. The user can at any time moveback to the previous screen by pressing “cancel” (top right of allscreens). Alternatively, the facility to select and fix appointmentswith a physician (as described above) is automatically presented to theuser (rather than via pressing of button 907) after clicking on button905, and completing the triaging questionnaire, as noted next.

In FIG. 9B, pressing button 905 leads the user to screen of FIG. 9C fortriaging questions. As indicated in FIG. 9C, a questionnaire ispresented with questions regarding specific health conditions (911, 912,913). The user can select one or more of the health conditions, andpress button 914, upon which the user is presented with screen of FIG.9D. In FIG. 9D, health conditions are presented (915, 916, 917 and 918),which the user can select based on his health condition. Upon completionof the triaging questionnaire, the answers provided are transmitted tothe remote terminal where the remote physician will use it fordiagnostics. Additionally, the user is presented with options ofavailable healthcare professionals for consultation, as described abovewith respect to pressing of button 907 and the resulting screens.

In FIG. 9B, button 906 is for initiating a consultation with a physicianwith whom an appointment has already been fixed as, noted above.Alternatively, the consultation can be initiated upon completion offixing an appointment with a physician as noted above, and if thephysician is currently available for consultation. In either case, theuser is then presented with the screen of FIG. 9H. The consultation isenabled by the tele-health application placing an audio and/or videocall with the remote physician. Once the call is established, the useris presented with the screen of FIG. 9H.

The screen of FIG. 9H enables the user, under direction of the physicianat the remote end, to generate diagnostic data, such as heart/lungauscultation audio data using stethoscope 130, otoscopy data usingotoscope 150/600, diagnostic data relating to temperature, bloodpressure, blood oxygen content and heart rate from a finger-press on thecamera port, and also diagnostic data captured by external devices andobtained by OCR block 845, as described above, thereby facilitating acomplete medical examination.

Specifically, pressing of button 941 (with stethoscope's (130)auscultation piece 130A placed on heart area of the user), initiatescapture of heart auscultation audio data via stethoscope 130, with theuser being presented with screen of FIG. 9I. FIG. 9I indicates that theheart auscultation data is being filtered by ‘Filter I’, which isexecuted by heart auscultation audio block 810 (FIG. 8 ).

Pressing of button 942 (with stethoscope's (130) auscultation piece 130Aplaced on lung area of the user), initiates capture of lung auscultationaudio data via stethoscope 130, with the user being presented withscreen of FIG. 9J. FIG. 9J indicates that the lung auscultation data isbeing filtered by ‘Filter II’, which is executed by lung auscultationaudio block 810 (FIG. 8 ).

Pressing of button 943 (with otoscope's (150) speculum 150A/610 placedon ear/nose/throat of user, and optionally pressing the flash of thecamera when otoscope 600 is used), initiates capture of video data viaotoscope 150, with the user being presented with screen of FIG. 9K. FIG.9K indicates that video data is being filtered by ‘Filter III’, which isexecuted by otoscopy processing block 835 (FIG. 8 ). The screen alsodisplays parameter 1, the temperature of the patient (951) determined byblock 840.

Pressing of button 944 (with otoscope's (150) speculum 150A/610 placedon ear/nose/throat of user, and optionally pressing the flash of thecamera when otoscope 600 is used), initiates capture of video data viaotoscope 150, with the user being presented with screen of FIG. 9L. FIG.9L indicates that video data is being filtered by ‘Filter IV’, which isexecuted by otoscopy processing block 835 (FIG. 8 ). The screen alsodisplays parameter 2 (Blood Pressure) (961), parameter 3 (pulse oxygencontent) (962), and parameter 4 (pulses per minute) (963) determined byblock 840.

It is noted here that the tele-health application provided by thepresent disclosure can generate various other screens also for displayof various other options and information to the user, and the screens ofFIGS. 9A-9L are merely representative. Further, a counterpartapplication is provided by the present disclosure and is installed inthe remote device to complement the operation of the modules (ortele-health application) in phone 110 and to enable the two-wayaudio/video consultation described herein. Such application can beimplemented in a known way by one skilled in the relevant arts uponreading the disclosure herein.

Several other features of the tele-health application not describedabove are summarized below, some of which have also been noted in othersections of this disclosure:

(A) Regulatory Compliance

-   -   1. HIPAA (Health Insurance Portability and Accountability        Act)-compliant patient and provider registration portal with two        factor authentication.    -   2. HIPAA-compliant patient and provider login web portal hosted        on cloud or private cloud or private server machines.    -   3. HIPAA-compliant login to androßid and iOS apps with below        functionality:        -   1. Login using ID/Email and password/fingerprint/face ID        -   2. Online mode            -   i. Both Video Calling and diagnostic features will be                accessible            -   ii. The ability to use additional features while in a                two way conference call        -   3. Offline mode            -   i. Only the diagnostics features are accessible with                on-device archiving and delayed relay ability to the                remote location (No video calling)

(B) Appointment and User Management Features

-   -   1. Two modes for the login        -   1. Patient mode            -   1. In patient mode when an appointment is made, a link                is sent via e-mail and added to the calendar in the                phone, by clicking the link the two clients can connect                via video call.        -   2. Provider mode            -   1. Review the appointment calendar            -   2. Appointment calendar to have active links which when                clicked would start the WebRTC video call

(C) Video Call Features

-   -   1. Mute microphone    -   2. Switch off front-side camera thereby permitting only        audio-call.    -   3. Switch between the back and front camera    -   4. Zoom in and out the cameras.    -   5. Switch to otoscope mode in which the object under otoscopy is        zoomed into and displayed on the local screen as well as remote        screen.    -   6. Switch between three different audio filters for real        time/Synchronous transmission of the auscultation audio        seamlessly over WebRTC or equivalent technology.

(D) Additional Features

-   -   1. If an earphone with a microphone is connected to the phone,        then the tele-health application generates an error message        saying that the additional features (stethoscope and otoscope)        cannot be used.    -   2. In offline mode audio is not sent to the remote device via        WebRTC, but made available on the earphones (in ear-buds type,        approved earphones).    -   3. In offline mode audio can be recorded and saved on the phone        or the cloud or emailed only via tele-health-app.    -   4. The saved audio, other user entered data will have meta-data        of the user name, birthdate, time, mode (clean channel, filter 1        or filter 2) and date of the recording.    -   5. User is allowed to replay the recordings.    -   6. User is allowed to delete the recordings.    -   7. User is warned before deleting the recordings (or any other        user data) that they can't be retrieved    -   8. There are app buttons to switch between different filters.        The filters are as follows        -   1. Clean channel with no filtering        -   2. Heart mode auscultation        -   3. Lung mode auscultation    -   9. Finger press of the patient on the back-side camera is        followed by a user input (button press) on a corresponding        screen (not shown).

Example internal details of phone 110 are described next.

5. Phone

FIG. 10 is a block diagram illustrating the implementation details ofmobile/smart phone 110 in an embodiment of the present disclosure.Although noted as a mobile phone, device 110 may in general beimplemented as a digital processing device having wireless or wiredcommunication capabilities, such as for example, tablet PC (personalcomputer), iPads, internet-connected chrome books, computers, IoT(Internet of Things) enabled devices as well as wired telephones, IP(internet protocol) phones, etc., as also note above.

Mobile phone 110 is shown containing battery 1001, power supply 1005,microphone interface 1010, camera interface 1015, processing block 1020,non-volatile memory 1030, random access memory (RAM) 1040, input block1050, display 1060, transmit chain 1070, receive chain 1080, switch 1090and antenna 1095. The specific components/blocks of mobile phone 1000are shown merely by way of illustration. However, mobile phone 1000 maycontain more or fewer components/blocks.

Battery 1001 in conjunction with power supply 1005 provides a regulatedpower supply voltage which powers each of blocks 1010, 1015, 1020, 1030,1040, 1050, 1060, 1070 and 1080. However, in FIG. 10 , only the powerconnection to processing block 1020 is shown for clarity.

Microphone interface 1010 receives audio signals on path 180 (microphoneport), amplifies the signals, and generates digital data representingthe signals using ADC. Microphone interface 1010 forwards the digitalaudio data to processing block 1020 for further processing.

Camera interface 1015 receives light (visible/infra-red, etc.) viacamera port 160, and generates digital video and/or still images in aknown way. Thus, for example, camera interface 1015 may contain RGBfilters, image sensor (e.g., CMOS), ADC, formatting circuits, etc., togenerate videos/images in the form of sets of RGB values. Camerainterface 1015 forwards the RGB values to processing block 1020.

Input/output block 1050 represents one or more input devices and outputdevices used to provide user inputs to mobile phone 1000, and outputdata from mobile phone 110 to a user. Thus, input/output block 1050 mayinclude a keypad as input and DAC, power amplifiers andspeakers/earphones as output. Display 1060 represents a display screen(e.g., liquid crystal display) to display images/text generated byprocessing block 1020.

Antenna 1095 operates to receive from, and transmit to, a wirelessmedium, information-bearing wireless signals. Switch 1090 may becontrolled by processing block 1020 (connection not shown) to connectantenna 1095 either to receive chain 1080 via path 1098, or to transmitchain 1070 via path 1079, depending on whether mobile phone 1000 is toreceive or transmit wireless signals.

Transmit chain 1070 receives data/speech/audio/video (in generalinformation signal, including those generated by transmit block 890 whenexecuted by processing block 1020) transmitted from processing block1020, generates a radio frequency (RF) signal modulated by theinformation signal according to corresponding standards such as GSM,CDMA, etc., and transmits the RF signal via switch 1090 and antenna1095. Receive chain 1080 receives an RF signal bearing an informationsignal (including signals representing data from the remote device notedherein) via switch 1090, path 1098 and antenna 1095, demodulates the RFsignal, and provides the extracted information (data/speech/audio/video)to processing block 1020.

Non-volatile memory 1030 is a non-transitory machine readable storagemedium storing instructions, which when executed by processing block1020, causes mobile phone 1000 to provide several features describedherein. Thus, non-volatile memory 1030 may store instructionsrepresenting the application modules noted herein, including those ofFIG. 8 . Non-volatile memory 1030 also stores data, such as that on path881 (FIG. 8 ). RAM 1030 is a volatile random access memory, and may beused for storing instructions and data.

Processing block 1020 (or processor in general) may contain multipleprocessing units (processors) internally, with each processing unitpotentially being designed for a specific task. Alternatively,processing block 1020 may contain only a single general-purposeprocessing unit. Processing block 1020 may execute instructions storedin non-volatile memory 1030 or RAM 1040 to enable mobile phone 110 tooperate to provide various features described herein. Specifically,processing block 1020 executes instructions contained in the tele-healthapplication, including the application modules of FIG. 8 , modules thatgenerate the screens of FIGS. 9A-9L, the application module controlblock, etc., to provide a two-way audio and/or video consultationbetween a user and a remote medical professional or physician inreal-time. Paths 821, 881, 891, 861, 801 and 831 of FIG. 8 are containedin paths 1052, 1032, 1027, 1062, 1021 and 1025 respectively.

6. Conclusion

References throughout this specification to “one embodiment”, “anembodiment”, or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present disclosure. Thus,appearances of the phrases “in one embodiment”, “in an embodiment” andsimilar language throughout this specification may, but do notnecessarily, all refer to the same embodiment.

While in the illustrations of FIGS. 1-4, 5A, 5B and 10 , althoughterminals/nodes are shown with direct connections to (i.e., “connectedto”) various other terminals, it should be appreciated that additionalcomponents (as suited for the specific environment) may also be presentin the path, and accordingly the connections may be viewed as being“electrically coupled” to the same connected terminals.

While various embodiments of the present disclosure have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. Thus, the breadth and scope of thepresent disclosure should not be limited by any of the above-describedembodiments, but should be defined only in accordance with the followingclaims and their equivalents.

What is claimed is:
 1. An A tele-health apparatus for use in combinationwith a telephone having a microphone and a camera, said tele-healthapparatus for facilitating remote examination of a living-being, thetele-health apparatus comprising: an auscultation piece to acquiresounds made by body of said living-being; an otoscope to obtain imagesof a part of said living-being; an adjuster; a solid medium coupled at afirst end to said auscultation piece; and a mechanical structure to gripsaid telephone and also attached to a second end of said solid medium,said second end being attached proximal to said microphone to cause saidacquired sounds to be transmitted from said auscultation piece to saidmicrophone, a first end of said adjuster also being physically attachedto said mechanical structure, and a second end of said adjuster beingattached to said otoscope, said adjuster being adjustable to place saidotoscope in front of said camera to enable said camera to receive saidimages from said otoscope, whereby said acquired sounds and said imagesare transmitted by said telephone to a remote person to facilitateremote examination of said living-being.
 2. The apparatus of claim 1,wherein said auscultation piece is part of a stethoscope, and said solidmedium is a windpipe of the stethoscope.
 3. The apparatus of claim 1,wherein said images comprise still pictures or videos of said part. 4.The apparatus of claim 3, wherein said adjuster is an X-Y adjuster,wherein said otoscope comprises a speculum attached to said X-Yadjuster, said X-Y adjuster being pivoted to said mechanical structurevia a pivot mechanism to place said otoscope in front of said camera inone position and away from said camera in a second position.
 5. Theapparatus of claim 2, wherein said solid medium has a sufficient lengthso that a user can simultaneously view and hear a physician at a remotedevice via said telephone, while simultaneously being able to operatesaid stethoscope.
 6. The apparatus of claim 3, wherein said camera is onthe back-side of said telephone, said otoscope further comprising lightguides to contain optic fibers, wherein said optic fibers are connectedto a flash light emitting diode (LED) of said telephone, therebyenabling capture of images from objects illuminated by said flash. 7.The apparatus of claim 3, wherein said otoscope contains a first lensand a first optical filter, said first lens to magnify the object to beviewed, wherein said first optical filter is designed to convertinfra-red light to visible light.
 8. The tele-health apparatus of claim1, wherein said second end of said solid medium comprises a couplingpiece, said coupling piece being coupled to said mechanical structurethrough a compression fit mechanism, said coupling piece also comprisingone or more suction cups or rubber couplers to allow a user's voice tobe captured by said microphone without significant attenuation, wherebysaid user can conduct audio communication also using said microphonewithout significant degradation.